System and method for joint source-channel encoding, with symbol, decoding and error correction

ABSTRACT

Disclosed is aA system and method for joint source-channel encoding, symbol decoding and error correction, preferably utilizing an arithmetic encoder with operational error detection space; and a combination sequential, and arithmetic, encoded symbol decoder structure.

This Application is a CIPThis is a reissue application of U.S. Pat. No.6,892,343 B2 issued May 10, 2005, U.S. Pat. No. 6,892,343 B2 issued fromU.S. application Ser. No. 09/816,398 filed Mar. 24, 2001, which claimsthe benefit under 35 U.SC. §119(e) of provisional application Ser. No.60/192,215 filed on Mar. 27, 2000.

TECHNICAL AREA

The present invention relates to source symbol encoding, decoding anderror correction capability in the context of noisy channels inelectronic communication systems. More particularly the prefered presentinvention is a system and method for joint source-channel encoding andvariable length symbol decoding with error correction, comprisingarithmetic encoder and combination sequential and arithmetic encodedsymbol, decoder means.

BACKGROUND

With the increasing popularity of mobile communications there has comerenewed interest in joint source-channel coding. The reason is thatshared mobile communications channels are restrictive in terms ofbandwidth and suffer from such as fading and interference etc., thusmaking some form of error protection essential, particularly wherevariable length codes are used. Further, it is well known that standardapproaches to error correction are expensive, in terms of requiredbandwidth, hence there exists a need for systems and methodology whichcan provide efficient source and channel encoding and symbol decodingwith error correction. Viable candidates include a joint source-channelencoding system and methodology which utilizes characteristics of asource, or source encoder, to provide error protection.

As background, it is noted that one of the earliest works that examinedthe effect of errors on variable length codes was that of Maxted andRobinson in an article titled “Error Recovery for Variable LengthCodes”, IEEE Trans. on Information Theory, IT-31, p. 794-801, (November1985). Corrections and additions to said work were provided by Monacoand Lawlor in “Error Recovery for Variable Length Codes”, IEEE Trans. onInformation Theory, IT-33, p. 454-456, (May 1987). And said work waslater extended by Soualhi et al. in “Simplified Expression for theExpected Error Span Recovery for Variable Length Codes”, Intl. J. ofElectronics, 75, p. 811-816, (November 1989), and by Rahman et al. in“Effects of a Binary Symetric Channel on the Synchronization Recovery ofVariable length Codes”, Computer J., 32, p. 246-251, (January 1989); aswell as by Takishima et al. in “Error States and SynchronizationRecovery for Variable Length Codes”, IEEE Trans. on Communications, 42,p. 783-792; as well as by Swaszek et al. in “More on the Error Recoveryfor Variable Length Codes”, IEEE Trans. on Information Theory, IT-41, p.2064-2071, (November 1995)., all of which focused mainly on theresynchronization ability of Huffman Codes.

In terms of joint source channel coding where the source and sourceencoder characteristics are used to provide error protection, one of theearliest works which incorporated variable length codes was that ofSayood, Liu and Gibson in “Implementation Issues in MAP JointSource/Channel Coding”, Proc. 22nd Annular Asilomar Conf. on Circuits,Systems, and Computers, p. 102-106, IEEE, (November 1988). Assuming aMarkov model for the source encoder output they used packetization toprevent error propagation and the residual redundancy at the sourceencoder output to provide error protection. This approach is used byPark and Miller who have developed a bit constrained decoderspecifically for use with variable codes, (see “Decoding Entropy-CodedSymbols Over Noisy Channels by MAP Sequency Estimation for AsynchronousHMMs”, Proc. Conference on Information Sciences and Systems, IEEE,(March 1999). Murad and Fuja, in “Robust Transmissions ofVariable-Length Encoded Sources”, Proc. IEEE Wireless and NetworkingConf. 1999, (September 1999); and Sayood, Otu and Demir in “JointSource/Channel Coding for Variable Length Codes”, IEEE Transactions onCommunications, 48:787-794, (May 2000), describe designs which make useof the redundancy at the source coder output for error correction.

The problem of low bandwidth hostile channels can also be addressedusing error resilent source codes which incorporate the possibility oferrors in the channel and provide mechanisms for error concealment. Workin the area includes that of Yang. et al. as reported in “Robust ImageCompression Based on Self-Synchronizing Huffman Code and Inter-SubbandDependency”, Proc. thirty-second Asilomar Conference on Signals, Systemsand Computers, p. 986-972 (November 1997), who use theself-synchronizing property of suffix rich Huffman codes to limit errorpropagation, and correlation between subbands to provide errorcorrection/concealment.

In addition, there exist a number of concatenated schemes in which thesource and channel encoders are concatenated in the traditional mannerwith channel resources allocated between them based on thecharacteristics of the channel. If the channel is very noisy, more bitsare allocated to the channel and fewer to source encoding, and thesituation is reversed when the channel conditions are more favorable.Examples of this approach include the work of Regunathan et al. aspresented in an article titled “Robust Image Compression for TimeVarying Channels”, Proc. Thirty-first Asilomar Conf. on Signals, Systemsand Computers, p. 968-972, (November 1997) and in an article titled“Progressive Image Coding for Noisy Channels”, by Sherwood et al., IEEESignal Processing Lett., 4 p. 189-191, (July 1997).

Most of the schemes referenced above use Huffman coding or variantsthereof as the variable length coding scheme, however, with theincreasing popularity of arithmetic coding, there has developed interestin joint source channel coding schemes which use said arithmetic coding.One such approach is described in “Arithmetic Coding Algorithm withEmbedded Channel Coding”, ElMasry, Electronics Lett., 33 p. 1687-1688,(September 1997); and another is described in “Integrating ErrorDetection into Arithmetic Coding”, Boyd et al., IEEE Transactions onCommunications, 45(1), p. 1-3, (January 1997). The ElMasry approachinvolves generation of parity bits which are embedded into arithmeticcoding procedure for error correction. The Boyd approach showed that byreserving probability space for a symbol which is not in the sourcealphabet the arithmetic code can be used for detecting errors. Reservingprobability space for a symbol that will never be generated means thatless space remains for the source alphabet and this translates into ahigher coding rate. Said overhead, however, is small considering thecapability of error detection enabled, as described by Kozintsev et al.in “Image Transmission Using Arithmetic Coding Based on Continuous ErrorDetection”, Proc. of Data Compression Conf. p. 339-348, IEEE ComputerSociety Press, (1998) regarding two scenarios, (eg. Automatic RepeatRequest (ARQ) based communications and serially concatenated codingschemes with an inner error correction code and an outer error detectioncode), which use error detecting capability of the arithmetic code withan error detection space.

With an eye to the present invention a Key-word Search for relevantPatents which involve inner and outer coding, trellis coding, datacompression, error detection, error correction, variable length coding,arithmetic coding, and data transmission over noisy channels, hasprovided:

-   U.S. Pat. No. 5,200,962 to Kao et al. is disclosed as it describes    data compression with error correction.-   U.S. Pat. No. 5,745,504 to Bang is disclosed as it describes bit    error resilient variable length code transmission.-   U.S. Pat. No. 6,009,203 to Liu et al. is disclosed as it describes    variable length coding.-   U.S. Pat. No. 5,233,629 to Paik et al. is disclosed as it concerns    trellis coded quadrature amplitude modulation.-   U.S. Pat. No. 4,862,464 to Betts et al. is disclosed as it describes    error detection in digital modems using trellis coding.-   U.S. Pat. No. 5,206,864 to McConnell is disclosed as it concerns    optimization of error correction, inner and outer coding.-   U.S. Pat. No. 5,841,794 to Inoue et al. is disclosed as it describes    error correction for digital data.-   U.S. Pat. No. 5,910,967 to Vanderaar is disclosed as it describes    concatenated coding in which an inner code is configured to match    the needs of an outer code.-   U.S. Pat. No. 5,870,405 to Hardwick et al. is disclosed as it    describes digital communication over noisy channels.-   U.S. Pat. No. 5,517,511 to Hardwick et al. is disclosed as it    describes transmission of digital data over noisy channels.-   U.S. Pat. No. 5,983,382 to Pauls is disclosed as it concerns outer    coding.    Patents disclosed because they discuss Arithmetic coding are:    -   U.S. Pat. No. 5,311,177 to Kimura et al.;    -   U.S. Pat. No. 5,587,710 to Choo et al.;    -   U.S. Pat. No. 5,418,863 to Ando;    -   U.S. Pat. No. 4,295,125 to Langdon, Jr;    -   U.S. Pat. No. 4,286,256 to Langdon, Jr. et al;    -   U.S. Pat. No. 5,774,081 to Cheng et al.;    -   U.S. Pat. No. 5,715,332 to Nakanishi;    -   U.S. Pat. No. 5,710,826 to Osawa et al.;    -   U.S. Pat. No. 5,317,428 to Osawa et al.

No known reference or combination of references, however, discloses useof a joint source-channel encoding, symbol decoding and error correctionsystem comprising encoder means, modulation-transmission means, andcombination sequential, and encoded symbol, decoding means; whereinerrors detected by the encoded symbol decoding means are corrected bymethodlogy involving the changing of bistable elements in saidsequential decoder means, or selection of a series of sequential bitsfrom a plurality of said serieses of sequential bits which result fromchanging bistable elements in said sequential decoder means,particularly where said encoder means is an arithmetic encoder andencoded symbol decoding means comprises arithmetic decoder, and encodedsymbols are of variable length.

DISCLOSURE OF THE INVENTION

The present invention can be characterized as a system and methodinvolving a concatenated scheme in which the functional roles of both:

-   -   outer code error detection; and    -   inner code error correction;        are facilitated by a joint source-channel encoder which        comprises operational reserved symbol probability space. In a        general preferred embodiment, outer code error detection is        performed by an encoded symbol decoder means, and inner code        error correction is achieved utilizing a sequential decoder        means which uses information from both a channel output and the        encoded symbol decoder means. In the preferred embodiment the        symbol encoder means is an arithmetic encoder means and the        encoded symbol decoder means is an arithmetic decoder means with        error detection capability that functions by detecting the        presence of encoded reserved symbol(s), which are not allowed as        input to said source-channel encoder means.

The present invention system can be described as a variable symbollength, joint source-channel encoding, symbol decoding and errorcorrection system comprising:

-   -   encoder means;    -   modulation-transmission means; and    -   combination sequential, and encoded symbol,    -   decoding means;        wherein the encoder means comprises input means for accepting a        sequential plurality of “allowed input symbols” and output means        for outputting an encoded sequence of bits for allowed symbols        input thereinto; where the terminology “allowed input symbols”        indicates symbols which when detected by the decoding means will        not typically identify that an error has occurred, (ie. they are        a part of an allowed alphabet of input symbols). Further, the        symbol encoder means is functionally interconnected to said        modulation-transmission means such that entry of a symbol to        said encoder means results in said encoder means outputting an        encoded sequence of bits therefore into said        modulation-transmission means, and said modulation-transmission        means and combination sequential, and encoded symbol, decoding        means are functionally interconnected such that an encoded        symbol sequence of bits entered to said modulation-transmission        means enters said combination sequential, and encoded symbol,        decoding means. Importantly, said sequential decoding means        comprises a plurality of bistable elements. Continuing, said        encoded symbol decoding means comprises means for initiating an        error correction routine to the end that, upon the encountering        the presence of an unexpected encoded reserved symbol:    -   at least one bistable element in said sequential decoding means        is changed; or    -   selection is made of a series of sequential bits, said selection        being made from a group consisting of a plurality of such series        of sequential bits which result from the changing of bistable        elements in said sequential decoder means;        wherein the terminology “reserved symbol” is used to identify a        symbol not allowed as an input symbol to said symbol encoding        means, and which, upon being detected by the encoded symbol        decoding means will trigger an error correcting routine.

Said encoder means can optionally further comprise means for generating,and in a sequence expected by said encoded symbol decoding means,outputting an encoded sequence of bits for at least one reserved symbolbefore and/or after an encoded

-   -   allowed input symbol, which reserved symbol is not allowed as an        input symbol to said encoder means input means. In use, the        present invention then provides that, upon the detecting of the        absence of an expected encoded reserved symbol, as well as upon        detecting the presence of an unexpected encoded reserved symbol:    -   at least one bistable element in said sequential decoding means        is changed; or    -   selection is made of a series of sequential bits, said selection        being made from a group consisting of a plurality of such series        of sequential bits which result from the changing of bistable        elements in said sequential decoder means.        (Note that the selections just recited correspond to serial and        parallel decoding error correction schemes, respectively).

Continuing, where the encoder means and encoded symbol decoder means arearithmetic, the present invention joint source-channel encoding,decoding and error correction system can be described as comprising:

-   -   arithmetic encoder means;    -   modulation-transmission means; and    -   decoding means;        wherein said arithmetic encoder means comprises input means for        accepting a sequential plurality of allowed input symbols and        output means for outputting an encoded sequence of bits for        allowed symbols input thereinto, and wherein said decoding means        comprises a functional combination of:    -   a sequential decoder means which comprises a sequence of        bistable elements, each of which can be set to represent encoded        symbol bits, and    -   an arithmetic decoder means.        Further, said arithmetic encoder means is functionally        interconnected to said modulation-transmission means and said        modulation-transmission means is functionally interconnected to        said decoding means. In use, said arithmetic encoder means is        caused to receive a sequence of allowed symbols at its input        means and provides an encoded sequence of bits for at least some        thereof, said sequence of bits being caused to arrive at said        decoding means. Continuing, said arithmetic decoder means has        error detection means such that in use said arithmetic decoder        means, upon detecting the presence of an unexpected encoded        reserved symbol, initiates an error correction routine to the        end that:    -   at least one bistable element in said sequential decoding means        is changed; or    -   selection is made of a series of sequential bits, said selection        being made from a group consisting of a plurality of such series        of sequential bits which result from the changing of bistable        elements in said sequential decoder means.        As described with respect to the first recitation of a present        invention system, said reserved symbol is not allowed as an        input symbol to said symbol encoding means, and the selections        just recited correspond to sequential and parallel error        correction schemes, respectively.

Where the encoder means and encoded symbol decoder means are arithmetic,the present invention joint source-channel encoding, decoding and errorcorrection system can be more precisely described as comprising:

-   -   arithmetic encoder means;    -   modulation-transmission means; and    -   decoding means;        wherein said decoding means comprises a functional combination        of a sequential decoder means which comprises a sequence of        bistable elements, each of which can be set to represent encoded        symbol bits, and an arithmetic decoder means. Further, said        arithmetic encoder means comprises input means for accepting a        sequential plurality of allowed input symbols and output means        for outputting an encoded sequence of bits for allowed symbols        input thereinto, and said arithmetic encoder means further has        means for generating and, in a sequence expected by said        arithmetic decoder means, outputting an encoded sequence of bits        for at least one reserved symbol before and/or after an encoded    -   allowed input symbol, which reserved symbol is not allowed as an        input symbol to said arithmetic encoder means input means. Said        arithmetic encoder means is functionally interconnected to said        modulation-transmission means and said modulation-transmission        means is functionally interconnected to said decoding means. In        use said arithmetic encoder means receives a sequence of allowed        symbols at its input means and provides an encoded sequence of        bits for at least some thereof in combination with a sequence of        bits which represent at least one encoded reserved symbol in a        pattern expected by said arithmetic decoder means, said sequence        of bits being caused to arrive at said decoding means.        Importantly, said arithmetic decoder means has error detection        means such that in use said arithmetic decoder means, upon        detecting:    -   the absence of an expected encoded reserved symbol; or    -   the presence of an unexpected encoded reserved symbol;        initiates an error correction routine to the end that:    -   at least one bistable element in said sequential decoding means        is changed; or    -   selection is made of a series of sequential bits, said selection        being made from a group consisting of a plurality of such series        of sequential bits which result from the changing of bistable        elements in said sequential decoder means.

A method of practicing the present invention, assuming the presence ofan arithmetic encoder and arithmetic encoded symbol decoding system, canbe recited as:

-   a. providing a present invention system as recited above;-   b. inputting a plurality of symbols to the input means of said    arithmetic encoder means;-   c. causing said arithmetic encoder means to encode at least some of    said plurality of symbols and output bits corresponding thereto,    optionally intermingled with at least one arithmetic encoder means    generated encoded reserved symbol, into said modulation-transmission    means;-   d. causing said modulation-transmission means to enter said at least    some of said plurality of encoded symbols along with at least one    encoded reserved symbol entered into said modulation-transmission    means, into said functional combination of said sequential decoder    means and arithmetic decoder means;-   e. causing said arithmetic decoder means to, if detecting    non-present expected or present unexpected encoded reserved    symbol(s):    -   change at least one bistable element in said sequential decoder        means; or    -   select a series of sequential bits, said selection being made        from a group consisting of a plurality of such series of        sequential bits which result from the changing of bistable        elements in said sequential decoder means.

It is noted that the arithmetic encoder means and decoding means, whichcomprises a sequential decoder means and an arithmetic decoder means,can be any electronic systems which perform the indicated function.

It is felt beneficial to provide insight to a specific error correctionprocedure which can be performed by the present invention. Again, apresent invention joint source-channel encoding system can be consideredto be sequentially comprised of:

-   -   an arithmetic encoder;    -   a modulation-transmission channel;    -   a sequential decoder means; and    -   an arithmetic decoder;        said arithmetic encoder feeding to said transmission channel,        and said transmission channel feeding to said sequential decoder        means, and said arithmetic decoder having access both from and        to said sequential decoder means. In use, a sequence of symbols        is entered into said arithmetic encoder with the result being        that said sequence of symbols is encoded and exited therefrom as        a binary bit stream sequence of +x√{square root over (Es)} and        −x√{square root over (Es)} signals, corresponding to a string of        “1”/(“0”)'s and “0”/(“1”)'s which pass through said transmission        channel and enter said sequential decoder means, (where said x        is a fraction). Hard logic circuit decisions are made as to the        presence of “1”/(“0”)'s and “0”/(“1”)'s based on said binary bit        stream sequence of +x √{square root over (Es)} and −x√{square        root over (Es)} signals while identify decisions based upon        signals wherein “x” is of a value so as to cause the values of        +x√{square root over (Es)} or −x√{square root over (Es)} to be        within a null zone of +Δ to −Δ around 0.0, and identifying said        decisions as “branch point” decisions in said sequential decoder        means. Further, monitoring output from said arithmetic decoder        results in detection of errors, and when an error is indicated        thereby, a “branch point” is identified in said sequential        decoder means and an inversion of “1”/(“0”) or “0”/(“1”) based        binary bit thereat to “0”/(“1”) or “1”/(“0”) is effected as an        attempt to correct said detected error.

A method of correcting errors in decoded symbols which are encoded by anarithmetic encoder in joint source-channel coding system, comprises thesteps of:

-   a. providing a present invention system as recited above;-   b. entering a sequence of symbols into said arithmetic encoder such    that said sequence of symbols are encoded and exited therefrom as a    binary bit stream sequence of +x √{square root over (Es)} and    −x√{square root over (Es)} signals, corresponding to a string of    “1”/(“0”)'s and “0”/(“1”)'s which pass through said transmission    channel and enter said sequential decoder means, where x is a    fraction;-   c. making hard logic circuitry decisions as to the presence of    “1”/(“0”)'s and “0”/(“1”)'s based on said binary bit stream sequence    of +x√{square root over (Es)} and −x√{square root over (Es)} signals    while identify decisions based upon signals wherein x is of a value    so as to cause the values of +x{square root over (Es)} or −x√{square    root over (Es)} to be within a null zone of +Δ to −Δ around 0.0, and    identifying said decisions as “branch point” decisions in said    sequential decoder means;    -   d. monitoring output from said arithmetic decoder for errors and        when an error is indicated thereby, identifying a “branch point”        in said sequential decoder means and correcting the “1”/(“0”) or        “0”/(“1”) based binary bit thereat by inverting it to “0”/(“1”)        or “1”/(“0”).

Said method of error correction can involve step d. being practiced morethan once, with said error correcting method further comprising the stepof:

-   e. defining a tolerable Hamming distance threshold Tc, and keeping    count of the number Kc of “branch points” in said sequential decoder    means at which correction of the “1”/(“0”) or “0”/(“1”) based binary    bit thereat by inverting to “0”/(“1”) or “1”/(“0”) has been    performed; and    -   if Kc exceeds Tc, expanding the null zone by increasing the        magnitude of Δ, thereby making available additional “branch        points”.

Further, said error correction method can further comprise the step of:

-   f. determining in a second or greater practice of step e. if the    identified “branch point” is sequentially prior to the “branch    point” identified in the immediately previous practice of step e.    and if so decreasing the value of Kc by 1, otherwise increasing the    value of Kc by 1.

Alternatively, said error correcting method can involve practice of stepd. more than once, with said error correcting method further comprisingthe step of:

-   e. defining a means for calculating a Euclidean distance between    received and decoded symbols, and a tolerable rate of increase of    Euclidean distance between sequential practice of step d., and    -   if said Euclidean distance increases faster than at said        tolerable rate, expanding the null zone by increasing the        magnitude of Δ, thereby making available additional “branch        points”.

Again, the error detection method, in step d., involves thedetermination of the presence or absence of non-alphabet, (ie.reserved), symbols other than as expected, said non-alphabet symbolsbeing not-allowed as arithmetic encoder input symbols.

The just described approach to correcting errors requires that“branch-points” in the sequential decoder means be determined based upona “null-zone” criteria, and involves retracking the contents of thesequential decoder means, and selectively changing an identified“1”/(“0”) to “0”/(“1”), when an error is identified. It is possible,however, to identify bistable elements in said sequential decoder meansand define them as fixed branch points, based upon the modulationtechnique utilized. For instance, if a trellis coded modulation schemeis utilized, a well known 8-PSK Constellation Codeword Assignmentapproach can be practiced. When such an approach to correcting errors indecoded symbols which are encoded by an arithmetic encoder in jointsource-channel coding system is utilized, the method thereof can bedescribed as comprising the steps of:

-   a. providing a joint source-channel encoding, decoding and error    correction, system comprising:    -   arithmetic encoder means;    -   modulation-transmission means; and    -   decoding means;        wherein said decoding means comprises a functional combination        of a sequential decoder means which comprises a sequence of        bistable elements, each of which can be set to represent encoded        symbol bits, and an arithmetic decoder means. Specific bistable        elements in said sequential decoder means are identified as        branch points based upon the modulation scheme utilized, (eg.        Trellis Coded Modulation and application of 8-PSK Constellation        Codeword Assignment Criteria). Said arithmetic encoder means        further comprises input means for accepting a sequential        plurality of allowed input symbols and output means for        outputting an encoded sequence of bits for allowed symbols input        thereinto, and said arithmetic encoder means further optionally        has means for generating and, in a sequence expected by said        arithmetic decoder means, outputting an encoded sequence of bits        for at least one reserved symbol before and/or after an encoded    -   allowed input symbol, which reserved symbol is not allowed as an        input symbol to said arithmetic encoder means input means. Said        arithmetic encoder means is functionally interconnected to said        modulation-transmission means and said modulation-transmission        means is functionally interconnected to said decoding means,        such that in use, said arithmetic encoder means receives a        sequence of allowed symbols at its input means and provides an        encoded sequence of bits for at least some thereof in optional        combination with    -   at least one encoded reserved symbol in a pattern expected by        said arithmetic decoder means, said sequence of bits being        caused to arrive at said decoding means. Said arithmetic decoder        means has error detection means such that in use said arithmetic        decoder means, upon detecting the absence of an expected encoded        reserved symbol, or the presence of an unexpected encoded        reserved symbol, initiates an error correction routine to the        end that:    -   selection is made of a series of sequential bits, said selection        being made from a group consisting of a plurality of such series        of sequential bits which result from the changing of bistable        elements in said sequential decoder means at said specific        branch points;        is performed. Said method continues with practice of the        following steps:-   b. entering a sequence of symbols into said arithmetic encoder means    such that said sequence of symbols are encoded and exited therefrom    as a binary bit stream sequence;-   c. monitoring output from said arithmetic decoder means for errors;-   d. when an error is detected by said arithmetic decoder means, (eg.    by detecting a present unexpected or non-present expected encoded    symbol), by using said specific branch point bistable elements in    said sequential decoder means, producing a plurality of series of    sequential bits which result from the changing of bistable elements    in said sequential decoder means at said specific branch points; and-   e. determining which series of sequential bits in said produced    plurality of series of sequential bits is most likely correct.

Said method of correcting errors in decoded symbols can, in step e.,involve determining which series of sequential bits in said producedplurality of series of sequential bits is most likely correct based onapplying at least on selection from the group consisting of:

-   a. eliminating any series of sequential bits which contains an    encoded reserved symbol;-   b. applying a metric to at least two series of sequential bits which    do not contain an encoded reserved sysbol, to determine which of    said at least two series of sequential bits is most likely correct;-   c. applying an Euclidean metric to at least two series of sequential    bits which do not contain an encoded reserved sysbol, to determine    which of said at least two series of sequential bits is most likely    correct.

Finally, it is specifically noted that, while not limiting, it isbelieved that Patentability is definitely established where the presentinvention system is comprised of an arithmetic encoder means, incombination with a decoding means which is comprised of a functionalcombination of a sequential decoder means and an arithmetic decodermeans, wherein in use, error correction methodology is initiated uponthe detecting, by the arithmetic decoder means, of a non-expectedencoded reserved symbol, or the absence of an expected encoded reservedsymbol sequentially inserted with encoded allowed symbols by thearithmetic encoder means. It is also noted that no arithmetic encodermeans is known which provides operational error detection space.Computer simulation thereof, and of sequential and arithmetic decodermeans then serve as example systems.

The present invention will be better understood by reference to theDetailed Description Section in combination with the Drawings.

SUMMARY OF THE INVENTION

It is therefore a primary purpose and/or objective of the presentinvention to provide a system comprising an outer symbol encoder meanswhich comprises operational error detection space, and a combinationsequential, and encoded symbol, decoding means, wherein said outerencoder means is preferably an arithmetic encoder, and the encodedsymbol, decoding means is preferably an arithmetic decoder.

It is another purpose of the present invention to disclose use ofreserved symbols as means to enable encoded symbol, decoding means,(arithmetic decoder), to identify errors, said identified errors beingcorrected by the changing of at least one bit is an associatedsequential decoder means.

It is another purpose yet of the present invention to teach that errordetection by an arithmetic decoder means can be based on detecting thepresence of an unexpected encoded symbol or on detecting the absence ofan expected encoded symbol.

It is yet another purpose of the present invention to disclose methodsof enhancing the operation of the sequential decoder means in correctingof errors involving distance calculations, (eg. Hamming and Euclideandistances).

It is a further purpose of the present invention to identify use of“null-zones”, or use of modulation technique determined specific “branchpoints” in a sequence of bistable elements in a sequential detectormeans.

Other purposes and/or objectives will become obvious from a reading ofthe Specification and Claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1a shows a preferred embodiment of the present invention systemapplied in a sequential decoding error correction scheme, said systemcomprising a concatenation of an Outer Arithmetic Encoder (1), (whichcomprises operational Error Detection Space); a Modulation-Transmissionmeans (2), (ie. a Channel); and a functional combination of an InnerSequential Decoder (3) and an Outer Arithmetic Decoder (4).

FIG. 1b demonstrates a present invention system which is similar to thatshown in FIG. 1a, but which is modified to emphasize application inpractice of a parallel decoding scheme.

FIG. 2 shows a “null zone” in a signal space for assumed binary BSPKsignalling in an additive white Gaussian noise channel.

FIG. 3 shows possible signal paths can be represented as a fullyconnected binary Trellis.

FIG. 4 shows a decoding tree for the FIG. 3 scenario, assuming errorpossibilities at each of the Branch Points.

FIGS. 5 and 6 show additional decoding trees which are described in thetext.

FIG. 7 shows and example of null zone expansion for Δ=0.1, Δ_(inc)=0.2,and L_(nz)=2.

FIG. 8a plots the number of symbols decoded vs. the number of decodeoperations, with the horizontal lines represent the locations of theerrors and the downward spikes indicate when the null zone is expandedfor a present invention system application.

FIG. 8b shows the final magnitude of the null zone denoted by the solid“stair step” line, showing the error locations and magnitudes.

FIG. 9 shows a plot of the fraction of paths that have not been prunedat each point for an application of a present invention scheme.

FIGS. 10a, 10b, 10c and 10d show how the decrease in the channel errorprobability, and corresponding drop in number of branch points, affectsthe required number of parallel decodings.

FIG. 11 shows comparison of the present invention Joint Source ChannelCoding Strategy to three conventional schemes, namely (4/5)convolutional codes, (one with 16 states and one with 8 states); and(223,255) Reed Solomon Code.

DETAILED DESCRIPTION

Turning now to the Drawings, there is shown in FIG. 1a, a preferredembodiment of the present invention system which is applied in asequential decoding error correction scheme. Shown are a concatenationof an Outer Arithmetic Encoder (1), (which comprises operational ErrorDetection Space); a Modulation-Transmission means (2), (ie. a Channel)and a functional combination of an Inner Sequential Decoder (3) and anOuter Arithmetic Decoder (4). Note said the Sequential (3) and theArithmetic (4) Decoders in combination are herein termed a Decodingmeans. Shown entering to the Arithmetic Encoder (1), (at input means foraccepting a sequential plurality of allowed input symbols), are Symbols(s_(n)), which Symbols (s_(n)) are members of an allowed alphabet. Shownexiting the Arithmetic Encoder (1), (at output means for outputting anencoded sequence of bits for allowed symbols input to said encodermeans), is a binary bit stream (x_(k)) which arrives at the SequentialDecoder (3) from Modulation-Transmission means (2) as a sequence ofbinary bits (r_(k)). Exiting said Sequential Decoder (3) is a sequenceof ({circumflex over (x)}_(k)), with ({circumflex over (x)}_(k))representing a sequence of hard decisions from the channel output basedon optimal symbol by symbol detection. A parameter d(r_(k) {circumflexover (x)}_(k)) is defined to be a distance, in some sense, between thedecoded and received signal at a time (k). Where convolution decodersare utilized this is often taken to be the Hamming Distance, (based onhard decisions ({circumflex over (x)}_(k)) between received and decodedsequences). The various possible decoded sequences can be viewed as abinary tree with the correct sequence being a path through said tree.

In use then the Arithmetic Encoder sends a binary stream of bits (x_(k))into the Modulation-Transmission means (2), in some mapped form, (eg.mapped to +/−√{square root over (Es)} for BPSK signaling).

(Note: BPSK stands for Binary Phase Shift Keying).

In the traditional sequential decoding scenario the structure of theconvolution code imposes a restriction on possible decoded sequencesand, hence, on possible branch points along a path. By discardingbranches in which an error has been detected, the decoding tree can bepruned such that what is left is the decoded sequence with the lowestHamming Distance from the received sequence. The structure of theconvolution code then defines the valid paths in the tree. The job ofthe Decoder is then primarily to find the valid path that results in adecoded sequence with the minimum distance from the received sequence.

Where Arithmetic Encoders are utilized, the situation is not as simple.To apply sequential decoding procedures to the case wherein anArithmetic Encoder is utilized, two considerations become important:

-   -   1. need for a condition enabling error detection, (a solution        thereof comprising a new and novel aspect of the present        invention); and    -   2. need for a decoding tree.

The first requirement is easily satisfied if use is made of errordetection space in the Arithmetic Encoder, and in fact, it is noted thatit is satisfied in a stronger manner than where convolution encoding isutilized. That is, in the Arithmetic Encoding case in which use is madeof error detection space, the appearance of a symbol corresponding tothe error detection space is a definite indication of error.

The second requirement is not as easily satisfied. This is becauseunlike in the convolution encoder case, the output of the arithmeticencoder is not restricted in terms of bit patterns which it can output,hence, an associated tree would have each bit as a branch point and thetree grows exponentially with the number of bits in a sequence. Thus itbecomes necessary to identify specific branch points which aremost-likely to be the location of error, and to arrive at a morerational code tree. Present invention methodology makes use ofinformation available at the output of the Modulation-Transmission means(2), (ie. a Channel), to obtain what are the most likely error branchpoints.

Assuming binary BSPK signalling and an additive white Gaussian noisechannel, the signal space can be represented as shown in FIG. 2.Further, “hard” decision receiver implementation decodes receivedsignals as a “0” or “1” depending on whether (r_(k)) is positive ornegative. Using the language of Gitlin and Ho in “Null Zone DecisionFeedback Equalizer Incorporating Maximum Likelihood Bit Detection:, IEEETrans. on Communications, 23 p. 1243-1250, (November 1975), a region ofwidth 2 Δ around the origin is termed the “null zone”. Received signals,the signal space projections of which land in said “null zone”, are,under present invention procedure, identified as branch pointcandidates, and it is noted that the number of so identified branchpoints can be increased or decreased by adjusting the magnitude of Δ.

The number of possible paths can be represented as a fully connectedbinary Trellis, such as shown in 3. The heavy lines in the FIG. 3Trellis indicate the path of a decoded signal, and the circled nodesindicate where the received signals fell into the null zone. The circlednodes, it is specifically noted, are the branch points, as defined inpresent invention methodology. FIG. 4 shows the FIG. 3 branch points ina decoding Tree representation. Further, it should be understood thatpresent invention methodology requires that the data represented by theheavy lines in FIG. 3 be stored in the Sequential Decoder (3) shown inFIG. 1a.

To aide with understanding, suppose that at point “X” in FIG. 3 an erroris detected. The present invention methodology then requires that datastored in the Sequential Decoder (3) be modified to attempt to removethe detected error. Present invention methodology would back up in theSequential Decoder (3) to the Bit identified as “Y”, which Bit “Y” isalso indicated in FIG. 4. proceeding in FIG. 3 the next Bit is a Branchpoint (BP1), which is again identified in FIG. 4. Note that twopossibilities exist at (BP1), and are represented in FIGS. 4 as (011)and (111). Similarly at (BP2) FIG. 4 shows that for each of the twobranches stemming from (011) and (111), two Branch Point possibilitiesexist, leading to the four possibilities (100) (000), (100) and (000)shown in FIG. 4. (Note that if a quadrature scheme were utilized, theTree of FIG. 4 would have four possibilities at each Branch Point (BP1)and (BP2)).

As a specific example, consider that the output of an Arithmetic Encoderis transmitted using a binary signalling scheme with √{square root over(E_(s=1))}. Further consider that said output is transmitted over aModulation-Transmission means (2), (ie. a Channel), which corrupts itwith additive noise such that the output of a signal receiver wouldprovide:R_(k)={−1.06, −1.06, −0.14, 1.56, −1.11, −1.39 . . . 0.09. 0.04. 0.67.−1.55, 1.03, 0.71}.If Δ=0.1 is chosen for the null zone magnitude, and hard decoding isperformed on the received values while marking bits corresponding tosignals that fall in the null zone by a “*”, then the following results:{circumflex over (X)}_(Δ=0.1)={0,0,0,1,0,0,*1,*1,1,0,1,1};and the Tree of FIG. 5 results. Decoding the received bits using anArithmetic Decoder (4) it is found that reading the sixth bit thedecodes the symbol corresponding to the error detection space. The factthat the symbol corresponding to the error detection space has beendecoded is denoted in FIG. 5 with an “X”. If the current path isdiscarded then the decoding has to be terminated. Therefore the path isretraced to the root of the Tree and the Δ is increased so that Δ=0.2.This causes one of the signals that was originally outside the null zoneto fall inside the null zone thus providing additional branch points sothat the decoded bit sequence becomes:{circumflex over (X)}_(Δ=0.2)={0,0, 1,1,0,0,*1,*1,1,0,1,1}

where denotes the explored branch point and the * denotes unexploredbranch points.

FIG. 6 shows the new decoding Tree corresponding to said binary string.The decoding begins at the root and the error is detected as before.However, this time there exists an alternative. Instead of progressingalong the upper branch, steps are retraced to the first branch point andthen a path along the lower branch is pursued. The new path is denotedin FIG. 6 as a dashed line. This time , after the 10^(th) bit an erroris detected. Hence, steps are retraced once again to the previous branchpoint and progress is pursued down the dotted line to find the decodedsequence.

In order to capture an error it is sufficient that Δ be greater than themagnitude of the error. It would seem then that selecting a large valuefor Δ is desirable, however, as already mentioned, such an approachleads to proliferation of branches in a resulting Tree. Further, it isknown that small magnitude errors are more likely than are largemagnitude errors, and as a result large values of Δ typically do notprovide significant benefit. Also, it is noted that the probability ofan error being within the last “n” symbols is:1−(1−ε)^(n)and as a result it is possible to keep the default value of small, andincrease it for signals corresponding to the last “n” symbols when anerror is detected. Another consideration is that the probablity of anerror being in a symbol close to the point at which an error is detectedis higher than the probability of the error being in a symbol furtheraway. With that in mind it is again noted that the reason for increasingthe value of Δ is to increase the number of branch points and that ifthe number of branch points is increased too far, computational time canbe wasted pursuing wrong paths. This leads to the insight that the nullzone magnitude can beneficially be adjusted in a discriminating manner,and an algorithm enabling this is:

-   -   let Δ_(inc) be the amount by which Δ is increased at each step,        and    -   let L_(nz) be the number of symbols over which it is proposed to        increase the value of Δ.        When an error is detected the path thereto is retraced for the        purpose of locating a branch point. If sufficient bits        corresponding to L_(nz) symbols are retraced then the null zone        is expanded by increasing Δ by Δ_(inc) for all signals        corresponding to the bits from the current position to the bits        corresponding to the farthest symbol forward where an error was        detected. Next the path is proceeded along forward with        arithmetic decoding. If an error is detected within the next        L_(nz) symbols retracing is pursued through bits corresponding        to 2L_(nz) symbols before expanding the null zone. This time the        null zone is incremented for all signals from the current        position to the farthest position where an error was detected by        Δ_(inc). Note that this has the effect of increasing the null        zone within L_(nz) symbols of the error detection by 2L_(nz). If        a correct path is still not available on the decoding tree and        steps corresponding to 3L_(nz) have been retraced, the null zone        is again incremented to all signals from the current location to        the farthest point forward where an error was detected by        Δ_(inc). This procedure is continued until the correct path is        found, or the computational burden exceeds a predetermined        threshold. In the later case a decoding error is declared. By        expanding the null zone in this manner the result is that null        zones at different signal locations are approximately        proportional to the probability of an error being detected, as        shown in FIG. 7. Further, if channel characteristics are known,        the probability of an error escaping the null zone can be        calculated. For instance, if the channel is an AWGN, (ie. Added        White Gaussian Noise), channel and the signal to noise ratio        over the channel is 10 dB then the probability of a hard        decision error is around 10⁻⁴. The probability of the error        falling outside of a null zone with Δ=0.4 √{square root over        (E_(s))} is about 10⁻¹⁴. If an acceptable packet loss rate for        the scheme is around 10⁻⁶ then an increase in the null zone        beyond 0.4 √{square root over (E_(s))} could not be justified.        Because of this condition a maximum null zone, denoted as        Δ_(max)(p), is chosen for a given probability of transmission        error.

It is noted that in an arithmetic decoder an error will almost alwayspropagate. However, the use of detection space essentially guaranteesthat any error will eventually be detected. The “Depth First” algorithmallows correction of the errors by exploring branches of a code tree,but said approach can become computationally expensive. It is, however,possible to prune a code tree in order to reduce the number ofcomputations. Several constraints can be used to accomplish said pruningthe code tree, and the inventors herein have made use of the fact thatmaking incorrect “corrections” causes increased deviation from a correctpath. Detection of proceeding along an incorrect path can beaccomplished by, for instance, keeping track of Hamming distance, and/orkeeping track of a Squared distance in the Euclidean sense.

Regarding the Hamming distance approach, keeping continuous track of thenumber of corrections still extant is key, with said count beingcompared against a threshold (T_(h)). The value of (T_(h)) is themaximum Hamming Distance between a received and decoded sequence whichit is decided can be tolerated. The reasoning is that the probability ofmore errors is less than the probability of fewer errors, and that if anadditional correction make the number of corrections extant greater than(T_(h)), then the null zone should be expanded by increasing the valueof Δ. Expanding the null zone increases the number of possible branchpoints and this increases the possibilities for decoding sequences at adistance (T_(h)) or less from the received sequence.

Regarding the approach based on Euclidean distance, a squared distancebetween received and decoded symbols is monitored. A running sum of thedistance between the sequential decoder output (x_(k)) and the receivedsequence is computed and compared to the distance between the output ofthe hard decision decoder (x_(k)) and the received sequence. At a time“n” this is accomplished by comparing the

Euclidean distance for the sequential decoder means:

$K_{e} = {\frac{1}{n}{\underset{k = 1}{\sum\limits^{m}}\left( {r_{k} - {\hat{x}}_{k}} \right)^{2}}}$where m is the encoded bit sequence length; with a threshold:

$T_{e} = {\frac{1 + \alpha}{N}{\underset{k = 1}{\sum\limits^{N}}\left( {r_{k} - {\hat{x}}_{k}} \right)^{2}}}$where α is an experimentally determined offset. The idea is that as harddecisions are changed the Euclidean distance between decoded andreceived sequences increases. If the distance increases at a high rateit can be detected and is indicative of proceeding down a wrong path. Ifa high rate of increase is detected the decoder takes the same action asit did for the case where (T_(h)) is exceeded under the Hammingapproach. If this approach becomes too restrictive, the offset which isconsidered acceptable can be incremented.

The value of (T_(h)) can be initialized to 1.0 if it is desired toexplore all single error events, with increases in (T_(h)) beingimplemented only when a maximum value of Δ is applied. It is noted,however, that single errors with large Δ may actually be less a problemthan double errors with a small Δ. Thus, it can be advantageous toincrease the value of (T_(h)) before increasing Δ.

In view of the foregoing, it should be appreciated that there are threeparameters which can be varied in controlling the discard criteria,namely:Δ; (T_(h)); and α.

In the following two present invention application scenarios arediscussed, namely Breadth First and Depth First. In the Depth Firstapproach the complexity depends almost completely on the number ofsymbol decodings that take place during a packet decoding. For a BreadthFirst approach, two major factors affect the complexity. The first isthe average number of decodings that take place during the decoding of apacket, which remains less than M times the number of symbols in a givenpacket. The second factor is the sorting that takes place before anexpansion at a branch point.

With the foregoing in mind, additional comments are appropriateregarding two distinguished approaches to Decoding, (ie. Breadth Firstand Depth First).

Breadth First Criteria

Applying the Breadth First approach, involves fixing the size of thenull zone prior to decoding. It is desirable to keep the null zone smallto reduce the number of branch points, and hence the amount ofcomputation, small. At the same time it is necessary to utilize a nullzone sufficiently large that the probability of missing an error isbelow what it is determined can be tolerated. Assuming an AWGN channelwith a known SNR, Δ can be selected as:

${\Delta\left( {p,q} \right)} = {\frac{Q^{- 1}\left( {1 - \left( {1 - q} \right)^{\frac{1}{m}}} \right)}{Q^{- 1}(p)} - 1.}$where m is the number of bits per packet, p is the channel errorprobability, and q is the desired lower bound on packet decoding rate.The function Q is given by:

${Q(x)} = {\int_{x}^{\infty}{\frac{1}{\sqrt{2\;\pi}}{\mathbb{e}}^{- \frac{y^{2}}{2}}{{\mathbb{d}y}.}}}$For this value of Δ the average number of branch points can becalculated as:

$\begin{matrix}{{B\left( {p,q} \right)} = {m{\int_{{({1 - \eta})}{Q^{- 1}{(p)}}}^{{({1 + \eta})}{Q^{- 1}{(p)}}}{\frac{1}{\sqrt{2\;\pi}}{\mathbb{e}}^{- \frac{y^{2}}{2}}{{\mathbb{d}y}.}}}}} & (2)\end{matrix}$which simplifies to:B(p, q)=m[Q((1−η)Q⁻¹(p))−Q((1+η)Q⁻¹(p))]  (3)where η=Δ(p,q) and m is the average number of bits per packet. In thisimplementation, detection of the error detection symbol by the decoderis used to prune the code tree, and the Euclidean distance between thedecoded and received sequence is used for selecting the best M paths.However, picking the value of M involves tradeoffs with larger values ofM increasing the probability that a correct path will be discarded. Thesolution adopted was to first perform decoding using a small value of M.If this does not result in successful decoding then M is increased by avalue M_(inc) and the procedure is repeated. Said procedure is repeateduntil the packet is decoded or a predetermined threshold M_(max) isreached, at which point a decoding error is declared.

It is further noted, in the context of a Breadth First approach, thatknowing the Modulation Technique applied can allow determination ofSpecifc Bistable Elements in a Sequential Decoder means which serve asfixed “Branch Points”. FIG. 1b demonstrates a present invention systemapplied to practice a method based in the alluded to approach. FIG. 1bis much like FIG. 1a, but note that a Modulator (5) is specificallyshown present between the Arithmetic Encoder (1) and Channel (2), andthat the Sequential Decoder of FIG. 1a is identified as furthercomprising a Demodulator (Demod). Note also that a binary bit stream(x_(k)) enters Modulator (5), but that a modulated binary bit stream(y₁) enters the Channel (2) with the sequential signals entered to theDemod/Sequential Decoder being identified by (r₁), rather than (x_(k)).It is not felt necessary to provide additional Figures to aide withunderstanding of the effects of this. Rather, referral to FIGS. 3-5shows that various series of sequential bits result where bits arechanged at various Bistable Element “Branch Points”, and said variousseries of sequential bits can be demonstrated as Branches of a Tree. Itis possible to, when an error is determined by detection of the presenceof an unexpected, or the absence of an expected encoded reserved symbolby an Arithmetic Decoder means, to form a plurality of possibleseries(es) of sequential bits, followed by:

-   -   selection being made of a series of sequential bits, said        selection being made from a group consisting of a plurality of        such series of sequential bits which result from the changing of        bistable elements in said sequential decoder means, (ie. the        various branches of a Tree).

Of course the selected series of sequential bits will be determined byat least one criteria being met, said criteria being for instance:

-   a. any series of sequential bits which contains an encoded reserved    symbol is eliminated;-   b. a metric is applied to at least two series of sequential bits    which do not contain an encoded reserved sysbol, to the end that    determination is made regarding which of said at least two series of    sequential bits is most likely correct;-   c. an Euclidean metric is applied to at least two series of    sequential bits which do not contain an encoded reserved sysbol, to    the end that determination is made regarding which of said at least    two series of sequential bits is most likely correct. Where this    approach is embodied, a suitable decoder, (ideal for use with system    which utilizes Trellis Coded Modulation (TCM)), is a List Viterbi    decoder where the size of the list can be adjusted to provide    increased error protection, with, of course, increased complexity.    The Kozintsev et al. article titled “Image Transmission Using    Arithmetic Coding Based on Continuous Error Detection”, which was    cited in the Background Section, it is noted, reported utilizing    arithmetic coding for additional error correction in a primarily    serially concatenated inner convolutional code scheme. Kozintsev et    al., however, did not suggest abandoning punctured convolutional    codes as the basis of the bulk of their error correction    methodology.

Depth First Criteria

To implement the Depth First approach the parameters required are:

-   -   initial value of the null zone Δ ;    -   the value by which the null zone is to be incremented Δ_(inc);    -   the “backtrack” distance L_(nz);    -   the threshold to be used to prune (T_(h)); and    -   the parameter a to be used in constructing a threshold for        Euclidean distance.

It has been found useful to define two thresholds T_(h,t) and T_(h,w)for Hamming distance and two thresholds α_(t) and α_(w) for theEuclidean distance. The total Hamming distance between the decodedsequence and the sequence obtained by hard decision decoding to thethreshold T_(h,t) as previously described. The Hamming distance betweenthe decoded sequence on the code Tree and the sequence obtained by harddecision decoding in a sliding window of size L_(w) to the thresholdT_(h,w). The end point of the sliding window is the current bit. Asimilar procedure is used for the Euclidean distance. It is noted thatthe values of T_(h, t) and T_(h,w) are obtained using two estimates ofchannel noise variance, one for the entire received sequence σ_(t) ²,and one for the sliding window of size L_(w). The variance σ_(t) ² istranslated into a channel probability error “p”, and the two thresholdsare obtained as:T_(k,f)=np(1+4σ_(t)), T_(h,w)=L_(w)p(1+8σ_(w))In a specific case, the length of the sliding window L_(w) was set to50, and both the T_(h) parameters were set to a minimum default value of2. The T_(e) parameters were found by hard decision decoding to produceX and then setting the α_(t) and α_(w) to 0.2 and 2.0 respectively. Thevalue of Δ was initially set to 0.10 √{square root over (E_(s))}. Whenthe decoder backtracked a symbol distance of L_(nz)=5, the value of Δwas increased by Δ_(inc)=0.10 √{square root over (E_(s))} to a maximumof 0.70 √{square root over (E_(s))}. If the decoder backtracked to theroot of the code Tree, the values of T_(h,t) and T_(e,t)

(where: $T_{e,w} = {{{\frac{\left( {1 + 0.2} \right)}{N}{\sum\limits_{N}\left( {r_{k} - x_{k}} \right)^{2}}} + T_{e,t}} = {\frac{\left( {1 + 0.2} \right)}{L_{w}}{\sum\limits_{L_{w}}\left( \left( {k - x_{k}} \right)^{2} \right)}}}$were increased by 10% and the values of T_(h,w) and T_(e,w) wereincreased by 20%.

Computational effort was determined by computing the ratio of the totalnumber of decode operations performed by the decoder to the number ofsymbols transmitted. In the case where no errors occurred this ratio isone. When an error is detected, because of backtracks, the decodingscheme requires more decode operations than the number of symbolstransmitted resulting in a value greater than one. When said ratioexceeded 10³ a decoding failure was declared.

FIGS. 8a and 8b present an example of the progression of the decodingprocedure. FIG. 8a plots the number of symbols decoded vs. the number ofdecode operations. The horizontal lines represent the locations of theerrors and the downward spikes indicate when the null zone is expanded.The trace progresses upward in a linear fashion until an error isdetected. At that point the decoder backtracks. The is indicated by thedownward slope of the decode trace. The trace continues downward until abranch point is reached. Then the trace slopes upward once again untilanother error is detected or the sequence is terminated. Note that therecan be a lot of “jostling” around the errors until the correct branchpoint is found. Also note that the null zone is only expanded when theprogression backtracks for a symbol distance of L_(nz). FIG. 8b showsthe final magnitude of the null zone denoted by the solid “stair step”line. Also shown are the error locations and magnitudes. These are thepeaks along the bottom. To obtain a correct decoding, the null zoneregion must be expanded to encompass all the introduced errors.

Table 1 presents the results of using the depth first decoding approachin terms of packet recovery rates for the four different values of theerror detection space:

-   -   (ε=0.5, 0.29, 0.16, 0.08).

TABLE 1 Packet Recovery Rates for Depth First Decoding p_(c) = 10^(−1.5)p_(c) = 10^(−2.0) p_(c) = 10^(−2.5) p_(c) = 10^(−3.0) NONE 0.00 0.000.01 24.64 ε = 0.08 0.00 0.39 46.63 96.72 ε = 0.16 0.00 17.04 95.9499.17 ε = 0.29 0.00 71.09 99.21 99.56 ε = 0.50 0.19 88.23 99.51 99.66For comparison, also included is the case where the standard arithmeticencoder is used, albeit with packetization. The results show a more than99% recovery rate for ε=0.16, 0.29, and 0.5; at a channel error rate of10⁻³ Similarly high results hold for ε=0.29 and 0.5 for p_(c)=10^(−2.5);where p_(e) is the probability of error for a symbol being transmittedover the channel.

However, for higher error rates the recovery rates drop significantly.Note that for a given channel error probability the amount of errorspace that is used is inversely proportional to the probability ofpacket loss.

To implement the Breadth First approach various parameter values wereselected as follows: M=200, M_(inc)=1800, and M_(max)=2000. Δ was chosento be {1.20, 1.00, 0.91, 0.82} for channel error probabilities of{10⁻¹⁵, 10⁻², 10^(−2.5), 10⁻³}, respectively. The parameters used givethe lower bounds on packet loss rates of {10^(−1.5)10⁻³, 10⁻⁴, 10⁻⁵},respectively. It should be recalled that he algorithm functions by firstlisting all possible paths at a branch point, then pruning all but the Mwhich are closest in Euclidean distance, to the received sequence.Between the branch point paths get pruned because progressing along themresults in the decoding of the error of the detection space.

FIG. 9 shows a plot of the fraction of paths that have not been prunedat each point. The average fraction of parallel decodings used is alsoindicated on FIG. 9 by the horizontal line. Note that more than half ofthe M paths are valid when a branch point is reached, the number ofbranches after the point is greater than M. The algorithm will thenprune the paths furthest from the received sequence. If the valid pathis ever pruned, the sequence will not decode at all or, with a smallprobability, it will decode incorrectly.

FIGS. 10a, 10b, 10c and 10d show how the decrease in the channel errorprobability, and corresponding drop in number of branch points, affectsthe required number of parallel decodings. Also note that the histogramfor ε=0.5 shifts quite significantly, dropping to around 50%. However,the distribution for ε=0.08 is concatenated on the left and thereduction on branch points has little effect. Using M=200 to decode thepacket results in being unable to decode the packet, hence the value ofM was increased to 2000. If thereafter the packet could still not bedecoded, a decoding failure was declared. For channel errorprobabilities of 10⁻³ and 10^(−2.5) a higher number of paths was usedfor less than one packet in 20,000 for all values of the error detectionspace. For a channel error probability of 10⁻² the same result held forε=0.5 and ε=0.29. For ε=0.16 the higher number of paths was used forabout 20% of the packets and about 7% of the packets could not bedecoded even with the higher number of paths. The situation wassignificantly worse for ε=0.08 with only 20% of the packets decodedusing M=200. For a channel error rate of 10^(−1.5) and ε=0.5, a littlemore that 20% of the packets were decoded using M=200 while about 50% ofthe packets were decoded with M=2000. For other values of almost all thepackets that could be recovered required M=2000.

Table 2 presents recovery rates for the case where Breadth Firstdecoding was applied.

TABLE 2 Packet Recovery Rates for Breadth First Decoding p_(c) =10^(−1.5) p_(c) = 10^(−2.0) p_(c) = 10^(−2.5) p_(c) = 10^(−3.0) NONE0.00 0.00 0.01 24.64 ε = 0.08 0.00 38.53 99.89 99.99 ε = 0.16 0.00 92.0399.94 99.99 ε = 0.29 16.63 99.30 99.95 99.99 ε = 0.50 73.24 99.33 99.9599.99The recovers rate is greater than 99% for all values of c for channelprobabilities of error of 10^(−2.5) and 10^(−3.0). For p_(e)=10⁻² therecovery rate is still greater than 99% for ε=0.29 and 0.5. Forp_(c)=10^(−1.5) a recovery rate of 73% of the packets for ε=0.50 maystill be useful for some applications. Notice that at higher error ratesthe Breadth first approach substantially out-performs the Depth Firstapproach. A penalty is paid for this performance at lower error rates,however, where the computational cost of the Breadth First approach ishigher than the Depth First approach.

Finally, performance of the present invention Joint Source ChannelCoding Strategy is compared to that of three conventional schemes:

-   -   (4/5) convolutional codes, (one with 16 states and one with 8        states); and    -   (223,255) Reed Solomon Code.        All these schemes have approximately the same rate overhead (the        exact number of information bits per data bit are shown in Table        3).

TABLE 3 Coding Rates$\text{Rate}\frac{\text{information~~bits}}{\text{data~~bits}}$ NONE1.000 ε = 0.29 0.901 ε = 0.50 0.819 (223, 255) RS 0.875 ⅘ Conv, s = 80.800 ⅘ Conv, s = 16 0.800The performances of the identified schemes is plotted in FIG. 11. Saidschemes are assumed to have the same header protection. Note that FIG.11 shows that the packet loss rat for two convolutional cases noted areconsiderably worse over the simulated range. Also note that the ReedSolomon code performs worse for low signal to noise ratios (SNR), butwhen the SNR increases to around 5.625, the Reed Solomon marginallyout-performs the present invention approach. The SNR's used in thesimulations were:

{2.368, 4.323, 5.714, 6.790} decibels.

Continuing, the amount of redundancy indicated in Table 3 shows that theconvolutional codes have the highest amount thereof, followed by thepresent invention scheme with ε=0/5. The present invention scheme withε=0.29 has the lowest amount of added redundancy of the schemescompared. It should be specifically appreciated that the presentinvention algorithm is only slightly more complex than a standardArithmetic encoding scheme, with the added complexity being presentprimarily at the decoder.

In the Depth First approach the complexity depends almost completely onthe number of symbol decodings that take place during a packet decoding,hence the complexity is slightly more than the average number of symboldecodings for a given SNR.

As alluded to earlier, for a Breadth First approach, two major factorsaffect the complexity. The first is the average number of decodings thattake place during the decoding of a packet, which remains less than Mtimes the number of symbols in a given packet. The averages can be seenin FIG. 11. The second factor is the sorting that takes place before anexpansion at a branch point. The average number of branch points can becalculated using Equation 3. At each branch point there exists thepossibility of needing to sort if the number of current paths is morethan half of the maximum. The complexity of the sort in the decodingused is Mlog₂(M). Thus, there is a large linear dependence, and anMlog(M) dependence in the decoding complexity.

Present invention schemes provide substantial packet recovery rates atchannel rates as low as 10^(−1.5) with low coding overhead. Such schemesare useful in hostile communication environments where minimal codingoverhead is advantageous. The approach may be especially useful formobile and wireless applications.

The present invention can be applied in communication systems whichoperate based on Binary Phase Shift Keying (BPSK), Quadrature PhaseShift Keying (QPSK) and Trellis Coded Modulation (TCM) etc.

It is noted that the terminology “variable length” refers to the lengthof code words assigned to input symbols, and the “joint source-channelsymbol encoding” refers the use of the same encoding means to encode“allowed alphabetic symbols” and “non-alphabet symbols” for use in errorcorrection.

Finally, it is noted that the present invention is primarily useful whenapplied with variable length symbol coding methods. For example Huffmancoding provides coding more probably symbols with shorter bit sequences.Arithmetic encoders code strings of symbols in a sequence of bits, andClaim language structure is focused to apply thereto.

Having hereby disclosed the subject matter of the present invention, itshould be obvious that many modifications, substitutions, and variationsof the present invention are possible in view of the teachings. It istherefore to be understood that the invention may be practiced otherthan as specifically described, and should be limited in its breadth andscope only by the Claims.

We claim:
 1. A variable length symbol, joint source-channel encoding,symbol decoding and error correction system comprising: encoder system;modulation-transmission means; and combination sequential, and encodedsymbol, decoding systems; said encoder system comprising input means foraccepting a sequential plurality of allowed input symbols and outputmeans for outputting an encoded sequence of bits for allowed symbolsinput thereinto; said encoder system being functionally interconnectedto said modulation-transmission means such that entry of a symbol tosaid encoder system results in said encoder means outputting an encodedsequence of bits therefore into said modulation-transmission means; saidmodulation-transmission means and combination sequential, and encodedsymbol, decoding systems being functionally interconnected such that anencoded symbol sequence of bits entered to said modulation-transmissionmeans enters said combination sequential, and encoded symbol, decodingsystems; said sequential decoding system comprising a plurality ofbistable elements; said encoded symbol decoding system comprising meansfor initiating an error correction routine to the end that, upon thedetecting of the presence of an unexpected encoded reserved symbol aselection from the group consisting of: at least one bistable element insaid sequential decoding means is changed; and selection is made of aseries of sequential bits, said selection being made from a groupconsisting of a plurality of such series of sequential bits which resultfrom the changing of bistable elements in said sequential decoder means;is performed; wherein said reserved symbol is not allowed as an inputsymbol to said symbol encoder system input means, and wherein thedetection of the presence of an encoded allowed symbol by said encodedsymbol decoding system does not initiate said error correction routine.2. A variable length symbol joint source-channel encoding, symboldecoding and error correction system comprising: encoder means;modulation-transmission means; and combination sequential, and encodedsymbol, decoding means; said encoder means comprising input means foraccepting a sequential plurality of allowed input symbols and outputmeans for outputting an encoded sequence of bits for allowed symbolsinput thereinto; said encoder means further having means for generating,and in a sequence expected by said encoded symbol decoding means,outputting an encoded sequence of bits for at least one reserved symbolbefore and/or after an encoded allowed input symbol, which reservedsymbol is not allowed as an input symbol to said encoder means inputmeans; said encoder means being functionally interconnected to saidmodulation-transmission means such that entry of a symbol to saidencoder means results in said encoder means outputting an encodedsequence of bits therefore into said modulation-transmission means; saidmodulation-transmission means and combination sequential, and encodedsymbol, decoding means being functionally interconnected such that anencoded symbol sequence of bits entered to said modulation-transmissionmeans enters said encoded symbol decoding means; said sequentialdecoding means comprising a plurality of bistable elements; said encodedsymbol decoding means comprising means for initiating an errorcorrection routine to the end that, upon the detecting of the presenceof an unexpected encoded reserved symbol, or the absence of an expectedencoded sequence of bits for a reserved symbol, a selection from thegroup consisting of: at least one bistable element in said sequentialdecoding means is changed; and selection is made of a series ofsequential bits, said selection being made from a group consisting of aplurality of such series of sequential bits which result from thechanging of bistable elements in said sequential decoder means; isperformed; and wherein the detection of the presence of an encodedallowed symbol, other than by its coincidental presence in the place ofan absent expected reserved symbol, by said encoded symbol decodingmeans, does not initiate said error correction routine.
 3. A jointsource-channel encoding, symbol decoding and error correction systemcomprising: encoder means; modulation-transmission means; and decodingmeans; wherein said encoder means comprises input means for accepting asequential plurality of allowed input symbols and output means foroutputting an encoded sequence of bits for allowed symbols inputthereinto; wherein said decoding means comprises a functionalcombination of a sequential decoder means which comprises a sequence ofbistable elements, each of which can be set to represent encoded symbolbits, and an encoded symbol decoder means; said encoder means beingfunctionally interconnected to said modulation-transmission means andsaid modulation-transmission means being functionally interconnected tosaid decoding means; such that in use said encoder means receives asequence of allowed symbols at its input means and provides an encodedsequence of bits for at least some thereof, said sequence of bits beingcaused to arrive at said decoding means; and said encoded symbol decodermeans having error detection means such that in use said encoded symboldecoder means, upon detecting the presence of an unexpected encodedsequence of bits for reserved symbol, initiates an error correctionroutine to the end that a selection from the group consisting of: atleast one bistable element in said sequential encoding decoder means ischanged; and selection is made of a series of sequential bits, saidselection being made from a group consisting of aplurality of suchseries of sequential bits which result from the changing of bistableelements in said sequential decoder means; is performed; wherein saidreserved symbol is not allowed as an input symbol to said encoder meansinput means, and wherein the detecting of the presence of an encodedallowed symbol by said decoding means does not initiate said errorcorrection routine.
 4. A joint source-channel encoding, symbol decodingand error correction system as in claim 3 in which said encoder means isan arithmetic encoder and said decoding means comprises, as the encodedsymbol decoder means, an arithmetic decoder.
 5. A joint source-channelencoding, symbol decoding and error correction system comprising:encoder means; modulation-transmission means; and decoding means;wherein said decoding means comprises a functional combination of asequential decoder means which comprises a sequence of bistableelements, each of which can be set to represent encoded symbol bits, andan encoded symbol decoder means; wherein said encoder means comprisesinput means for accepting a sequential plurality of allowed inputsymbols and output means for outputting an encoded sequence of bits forallowed symbols input thereinto, said encoder means further having meansfor generating and, in a sequence expected by said encoded symboldecoder means, outputting an encoded sequence of bits for at least onereserved symbol before and/or after an encoded allowed input symbol,which reserved symbol is not allowed as an input symbol to said encodermeans input means; said encoder means being functionally interconnectedto said modulation-transmission means and said modulation-transmissionmeans being functionally interconnected to said decoding means; suchthat in use said encoder means receives a sequence of allowed symbols atits input means and provides an encoded sequence of bits for at leastsome thereof in optional combination with a sequence of bits whichrepresent at least one encoded reserved symbol in a pattern expected bysaid decoder means, said sequence of bits being caused to arrive at saiddecoding means; and said encoded symbol decoder means having errordetection means such that in use said encoded symbol decoder means, upondetecting the absence of an expected encoded reserved symbol, or thepresence of an unexpected encoded reserved symbol, initiates an errorcorrection routine to the end that a selection from the group consistingof: at least one bistable element in said sequential decoding means ischanged; and selection is made of a series of sequential bits, saidselection being made from a group consisting of a plurality of suchseries of sequential bits which result from the changing of bistableelements in said sequential decoder means; is performed; and wherein thedetection of the presence of an encoded allowed symbol, other than byits coincidental presence in the place of an absent expected reservedsymbol, by said decoding means, does not initiate said error correctionroutine.
 6. A joint source-channel encoding, symbol decoding and errorcorrection system as in claim 5 in which said encoder means is anarithmetic encoder and said decoding means comprises, as the encodedsymbol decoder means, an arithmetic decoder.
 7. A joint source-channelencoding, symbol decoding and error correction system comprising:arithmetic encoder system; modulation-transmission means; and decodingsystem; wherein said arithmetic encoder system comprises input means foraccepting a sequential plurality of allowed input symbols and outputmeans for outputting an encoded sequence of bits for allowed symbolsinput therein to; wherein said decoding system comprises a functionalcombination of a sequential decoder system which comprises a sequence ofbistable elements, each of which can be set to represent encoded symbolbits, and an arithmetic decoder system; said arithmetic encoder systembeing functionally interconnected to said modulation-transmission meansand said modulation-transmission means being functionally interconnectedto said decoding system; such that in use said arithmetic encoder systemreceives a sequence of allowed symbols at its input means and providesan encoded sequence of bits for at least some thereof, said sequence ofbits being caused to arrive at said decoding system; and said arithmeticdecoder system having error detection means such that in use saidarithmetic decoder system, upon detecting the presence of an unexpectedencoded reserved symbol, initiates an error correction routine to theend that a selection from the group, consisting of: at least onebistable element in said sequential encoding means decoder system ischanged; and selection is made of a series of sequential bits, saidselection being made from a group consisting of a plurality of suchseries of sequential bits which result from the changing of bistableelements in said sequential decoder means decoder system; is performed;wherein said reserved symbol is not allowed as an input symbol to saidarithmetic encoder system input means, and wherein the detecting of thepresence of an encoded allowed symbol by said decoding system does notinitiate said error correction routine.
 8. A joint source-channelencoding, symbol decoding and error correction system comprising:arithmetic encoder means; modulation-transmission means; an decodingmeans; wherein said decoding means comprises a functional combination ofa sequential decoder means which comprises a sequence of bistableelements, each of which can be set to represent encoded symbol bits, andan arithmetic decoder means; wherein said arithmetic encoder meanscomprises input means for accepting a sequential plurality of allowedinput symbols and output means for outputting an encoded sequence ofbits for allowed symbols input thereinto, said arithmetic encoder meansfurther having means for generating and, in a sequence expected by saidarithmetic decoder means, outputting an encoded sequence of bits for atleast one reserved symbol before and/or after an encoded allowed inputsymbol, which reserved symbol is not allowed as an input symbol to saidarithmetic encoder means input means; said arithmetic encoder meansbeing functionally interconnected to said modulation-transmission meansand said modulation-transmission means being functionally interconnectedto said decoding means; such that in use said arithmetic encoder meansreceives a sequence of allowed symbols at its input means and providesan encoded sequence of bits for at least some thereof in optionalcombination with at least one encoded reserved symbol in a patternexpected by said arithmetic decoder means, said sequence of bits beingcaused to arrive at said decoding means; and said arithmetic decodermeans having error detection means such that in use said arithmeticdecoder means, upon detecting the absence of an expected encodedreserved symbol, or the presence of an unexpected encoded reservedsymbol, initiates an error correction routine to the end that aselection from the group consisting of: at least one bistable element insaid sequential decoding means is changed; and selection is made of aseries of sequential bits, said selection being made from a groupconsisting of a plurality of such series of sequential bits which resultfrom the changing of bistable elements in said sequential decoder means;is performed; and wherein the detection of the presence of an encodedallowed symbol, other than by its coincidental presence in the place ofan absent expected reserved symbol, by said decoding means, does notinitiate said error correction routine.
 9. A method of correcting errorsin decoded symbols which are encoded by an encoder means in a jointsource-channel coding system, comprising the steps of: a. providing ajoint source-channel encoding, symbol decoding and error correctionsystem comprising: encoder means; modulation-transmission means; anddecoding means; wherein said decoding means comprises a functionalcombination of a sequential decoder means which comprises a sequence ofbistable elements, each of which can be set to represent encoded symbolbits, and an encoded symbol decoder means; wherein said encoder meanscomprises input means for accepting a sequential plurality of allowedinput symbols and output means for outputting an encoded sequence ofbits for allowed symbols input thereinto, said encoder means furtherhaving means for generating and, in a sequence expected by said encodedsymbol decoder means, outputting an encoded sequence of bits for atleast one reserved symbol before and/or after an encoded allowed inputsymbol, which reserved symbol is not allowed as an input symbol to saidencoder means input means; said encoder means being functionallyinterconnected to said modulation-transmission means and saidmodulation-transmission means being functionally interconnected to saiddecoding means; such that in use said encoder means receives a sequenceof allowed symbols at its input means and provides an encoded sequenceof bits for at least some thereof, in optional combination with at leastone encoded reserved symbol in a pattern expected by said decoder means,said sequence of bits being caused to arrive at said decoding means; andsaid encoded symbol decoder means having error detection means such thatin use said encoded symbol decoder means, upon detecting the absence ofan expected encoded reserved symbol, or the presence of an unexpectedencoded reserved symbol, initiates an error correction routine to theend that a selection from the group consisting of: at least one bistableelement in said sequential decoding means is changed; and selection ismade of a series of sequential bits, said selection being made from agroup consisting of a plurality of such series of sequential bits whichresult from the changing of bistable elements in said sequential decodermeans; is performed; wherein the detection of the presence of an encodedallowed symbol, other than by its coincidental presence in th place ofan absent expected reserved symbol, by said decoding means, does notinitiate said error correction routine; b. inputting a plurality ofsymbols to the input means of said encoder means; c. causing saidencoder means to encode at least some of said plurality of symbols andoutput bits corresponding thereto into said modulation-transmissionmeans; d. causing said modulation-transmission means to enter said atleast some of said plurality of encoded symbols into said functionalcombination of said sequential decoder means and encoded symbol decodermeans; e. causing said encoded symbol decoder means to, if detecting apresent unexpected or absent expected, encoded reserved symbol, performa selection from the group consisting of: change at least one bistableelement in said sequential decoder means; and select a series ofsequential bits, said selection being made from a group consisting of aplurality of such series of sequential bits which result from thechanging of bistable elements in said sequential decoder means.
 10. Amethod as in claim 9, wherein said encoded reserved symbol is selectedfrom a group of at least two reserved symbols, and wherein at least twodifferent expected reserve symbols are selected and entered into saidmodulation-transmission means.
 11. A method of correcting errors indecoded symbols which are encoded by an arithmetic encoder in a jointsource-channel coding system, comprising the steps of: a. providing ajoint source-channel encoding, symbol decoding and error correctionsystem comprising: arithmetic encoder means; modulation-transmissionmeans; and decoding means; wherein said arithmetic encoder meanscomprises input means for accepting a sequential plurality of allowedinput symbols and output means for outputting an encoded sequence ofbits for allowed symbols input thereinto; wherein said decoding meanscomprises a functional combination of a sequential decoder means whichcomprises a sequence of bistable elements, each of which can be set torepresent encoded symbol bits, and an arithmetic decoder means; saidarithmetic encoder means being functionally interconnected to saidmodulation-transmission means and said modulation-transmission meansbeing functionally interconnected to said decoding means; such that inuse said arithmetic encoder means receives a sequence of allowed symbolsat its input means and provides an encoded sequence of bits for at leastsome thereof, said sequence of bits being caused to arrive at saiddecoding means; and said arithmetic decoder means having error detectionmeans such that in use said arithmetic decoder means, upon detecting thepresence of an unexpected encoded reserved symbol, initiates an errorcorrection routine to the end that a selection from the group consistingof: at least one bistable element in said sequential decoding means ischanged; and selection is made of a series of sequential bits, saidselection being made from a group consisting of a plurality of suchseries of sequential bits which result from the changing of bistableelements in said sequential decoder means; is performed; wherein saidreserved symbol is not allowed as an input symbol to said arithmeticencoder means input means, and wherein the detection of the presence ofan allowed encoded symbol by said decoding means does not initiate saiderror correction routine; b. inputting a plurality of symbols to theinput means of said arithmetic encoder means; c. causing said arithmeticencoder means to encode at least some of said plurality of symbols andoutput bits corresponding thereto into said modulation-transmissionmeans; d. causing said modulation-transmission means to enter said atleast some of said plurality of encoded symbols into said functionalcombination of said sequential decoder means and arithmetic decodermeans; e. causing said arithmetic decoder means to, if detecting apresent unexpected encoded reserved symbol perform a selection from thegroup consisting of: change at least one bistable element in saidsequential decoder means; and select a series of sequential bits, saidselection being made from a group consisting of a plurality of suchseries of sequential bits which result from the changing of bistableelements in said sequential decoder means.
 12. A method as in claim 11,wherein said encoded reserved symbol is selected from a group of atleast two reserved symbols.
 13. A method of correcting errors in decodedsymbols which are encoded by an arithmetic encoder in a jointsource-channel coding system, comprising the steps of: a. providing ajoint source-channel encoding, symbol decoding and error correctionsystem comprising: arithmetic encoder means; modulation-transmissionmeans; and decoding means; wherein said decoding means comprises afunctional combination of a sequential decoder means which comprises asequence of bistable elements, each of which can be set to representencoded symbol bits, and an arithmetic decoder means; wherein saidarithmetic encoder means comprises input means for accepting asequential plurality of allowed input symbols and output means foroutputting an encoded sequence of bits for allowed symbols inputthereinto, said arithmetic encoder means further having means forgenerating and, in a sequence expected by said arithmetic decoder means,outputting an encoded sequence of bits for at least one reserved symbolbefore and/or after an allowed input symbol, which reserved symbol isnot allowed as an input symbol to said arithmetic encoder means inputmeans; said arithmetic encoder means being functionally interconnectedto said modulation-transmission means and said modulation-transmissionmeans being functionally interconnected to said decoding means; suchthat in use said arithmetic encoder means receives a sequence of allowedsymbols at its input means and provides an encoded sequence of bits forat least some thereof in optional combination with at least one encodedreserved symbol in a pattern expected by said arithmetic decoder means,said sequence of bits being caused to arrive at said decoding means; andsaid arithmetic decoder means having error detection means such that inuse said arithmetic decoder means, upon detecting the absence of anexpected encoded reserved symbol, or the presence of an unexpectedencoded reserved symbol, initiates an error correction routine to theend that a selection from the group consisting of: at least one bistableelement in said sequential decoding means is changed; and selection ismade of a series of sequential bits, said selection being made from agroup consisting of a plurality of such series of sequential bits whichresult from the changing of bistable elements in said sequential decodermeans; is performed; wherein said reserved symbol is not allowed as aninput symbol to said arithmetic encoder means input means and whereinthe detection of the presence of an allowed symbol, other than by itscoincidental presence in the place of an absent expected encodedreserved symbol, by said decoding means, does not initiate said errorcorrection routine; b. inputting a plurality of symbols to the inputmeans of said arithmetic encoder means; c. causing said arithmeticencoder means to encode at least some of said plurality of symbols andoutput bits corresponding thereto, optionally intermingled witharithmetic at least one encoder means generated reserved symbol, intosaid modulation-transmission means; d. causing saidmodulation-transmission means to enter said at least some of saidplurality of encoded symbols, optionally along with at least one encodedreserved symbol entered into said modulation-transmission means, intosaid functional combination of said sequential decoder means andarithmetic decoder means; e. causing said arithmetic decoder means to,if detecting a non-present expected or present unexpected encodedreserved symbol, perform a selection from the group consisting of:change at least one bistable element in said sequential decoder means;and select a series of sequential bits, said selection being made from agroup consisting of a plurality of such series of sequential bits whichresult from the changing of bistable elements in said sequential decodermeans.
 14. A method as in claim 13, wherein said encoded reserved symbolis selected from a group of at least two reserved symbols, and whereinat least two different expected reserve symbols are selected and enteredinto said modulation-transmission means.
 15. A method of correctingerrors in decoded symbols which are encoded by an arithmetic encoder ina joint source-channel coding system, comprises the steps of: a.providing a joint source-channel encoding, symbol decoding and errorcorrection, system comprising: arithmetic encoder means;modulation-transmission means; and decoding means; wherein said decodingmeans comprises a functional combination of a sequential decoder meanswhich comprises a sequence of bistable elements, each of which can beset to represent encoded symbol bits, and an arithmetic decoder means;wherein said arithmetic encoder means comprises input means foraccepting a sequential plurality of allowed input symbols and outputmeans for outputting an encoded sequence of bits for allowed symbolsinput thereinto, said arithmetic encoder means further having means forgenerating and, in a sequence expected by said arithmetic decoder means,outputting an encoded sequence of bits for at least one reserved symbolbefore and/or after an encoded allowed input symbol, which reservedsymbol is not allowed as an input symbol to said arithmetic encodermeans input means; said arithmetic encoder means being functionallyinterconnected to said modulation-transmission means and saidmodulation-transmission means being functionally interconnected to saiddecoding means; such that in use said arithmetic encoder means receivesa sequence of allowed symbols at its input means and provides an encodedsequence of bits for at least some thereof in optional combination withat least one encoded reserved symbol in a pattern expected by saidarithmetic decoder means, said sequence of bits being caused to arriveat said decoding means; and said arithmetic decoder means having errordetection means such that in use said arithmetic decoder means, upondetecting the absence of an expected encoded reserved symbol, or thepresence of an unexpected encoded reserved symbol, initiates an errorcorrection routine to the end that at least one bistable element in saidsequential decoder means is changed; b. entering a sequence of symbolsinto said arithmetic encoder such that said sequence of symbols areencoded and exited therefrom as a binary bit stream sequence of +x√{square root over (E₈)} and −x √{square root over (E₈)} signals,corresponding to a string of “1”/(“0”)'s and “0”/(“1”)'s which passthrough said transmission channel and enter said sequential decodermeans, where x is a fraction; c. making hard logic circuitry decisionsas to the presence of “1”/(“0”)'s and “0”/(“1”)'s based on said binarybit stream sequence of +x√{square root over (E_(s))} and −x√{square rootover (E_(s))} signals while identify decisions based upon signalswherein x is of a value so as to cause the values of +x√{square rootover (E_(s))} or −x√{square root over (E_(s))} to be within a null zoneof +Δ to −Δ around 0,0, and identifying said decisions as “branchpoint”, decisions in said sequential decoder means; d. monitoring outputfrom said arithmetic decoder for errors and when if an error isindicated thereby, identifying a “branch point” in said sequentialdecoder means and correcting the “1”/(“0”) or “0”/(“1”) based binary bitthereat by inverting it to “0”/(“1”) or “1”/(“0”).
 16. A method ofcorrecting errors in decoded symbols as in claim 15, in which in step d.involves the determination of the presence or absence of non-alphabetsymbols other than as expected, said non-alphabet symbols beingnot-allowed as arithmetic encoder input symbols.
 17. A method ofcorrecting errors in decoded symbols as in claim 15, which comprisespracticing step d. more than once, with said error correcting methodfurther comprising the step of: e. defining a tolerable Hamming distancethreshold Tc, and keeping count of the number Kc of “branch points” insaid sequential decoder means at which correction of the “1”/(“0”) or“0”/(“1”) based binary bit thereat by inverting to “0”/(“1”) or“1”/(“0”) has been performed; and if Kc exceeds Tc, expanding the nullzone by increasing the magnitude of Δ, thereby making availableadditional “branch points”.
 18. A method of correcting errors in decodedsymbols as in claim 17, said error correction method of furthercomprising the step of: f. determining in a second or greater practiceof step e. if the identified “branch point” is sequentially prior to the“branch point” identified in the immediately previous practice of stepe. and if so decreasing the value of Kc by 1, otherwise increasing thevalue of Kc by
 1. 19. A method of correcting errors in decoded symbolsas in claim 15, which comprises practicing step d. more than once, withsaid error correcting method further comprising the step of: e. defininga means for calculating a Euclidean distance between received anddecoded symbols, and a tolerable rate of increase of Euclidean distancebetween sequential practice of step d., and if said Euclidean distanceincreases faster than at said tolerable rate, expanding the null zone byincreasing he magnitude of Δ, thereby making available additional“branch points”.
 20. A method of correcting errors in decoded symbolswhich are encoded by an arithmetic encoder in a joint source-channelcoding system, comprises the step of: a. providing a jointsource-channel encoding, symbol decoding and error correction systemcomprising: arithmetic encoder means; modulation-transmission means; anddeocding means; wherein said decoding means comprises a functionalcombination of a sequential decoder means which comprises a sequence ofbistable elements, each of which can be set to represent encoded symbolbits, and an arithmetic decoder means; specific bistable elements insaid sequential decoder means being identified as fixed branch points;wherein said arithmetic encoder means comprises input means foraccepting a sequential plurality of allowed input symbols and outputmeans for outputting an encoded sequence of bits for allowed symbolsinput thereinto, said arithmetic encoder means further having means forgenerating and, in a sequence expected by said arithmetic decoder means,outputting an encoded sequence of bits for at least one reserved symbolbefore and/or after an encoded allowed input symbol, which reservedsymbol is not allowed as an input symbol to said arithmetic encodermeans input means; said arithmetic encoder means being functionallyinterconnected to said modulation-transmission means and saidmodulation-transmission means being functionally interconnected to saiddecoding means; such that in use said arithmetic encoder means receivesa sequence of allowed symbols at its input means and provides an encodedsequence of bits for at least some thereof in optional combination withat least one encoded reserved symbol in a pattern expected by saidarithmetic decoder means, said sequence of bits being caused to arriveat said decoding means; and said arithmetic decoder means having errordetection means such that in use said arithmetic decoder means, upondetecting the absence of an expected encoded reserved symbol, or thepresence of an unexpected encoded reserved symbol, initiates an errorcorrection routine to the end that: selection is made of a series ofsequential bits, said selection being made from a group consisting of aplurality of such series of sequential bits which result from thechanging of bistable elements in said sequential decoder means at saidspecified branch points; is performed; b. entering a sequence of symbolsinto said arithmetic encoder means such that said sequence of symbolsare encoded and exited therefrom as a binary bit stream sequence; c.monitoring output from said arithmetic decoder means for errors; d. upondetection of an error by said arithmetic decoder means, producing aplurality of series of sequential bits which result from the changing ofbistable elements in said sequential decoder means at said branch pointsby using fixed branch point bistable elements in said sequential decodermeans; e. determining which series of sequential bits in said producedplurality of series of sequential bits is most likely correct utilizingat least one selection from the group consisting of:
 1. eliminating anyseries of sequential bits which contains an encoded reserved symbol; 2.applying a metric to at least two series of sequential bits which do notcontain an encoded reserved symbol, to determine which of said at leasttwo series of sequential bits is most likely correct;
 3. applying anEuclidean metric to at least two series of sequential bits which do notcontain an encoded reserved symbol, to determine which of said at leasttwo series of sequential bits is most likely correct.
 21. A system,comprising: a decoder, comprising an arithmetic decoder and a sequentialdecoder, configured to receive data from an arithmetic encoder via achannel, wherein the arithmetic decoder, upon detecting an absence of anexpected encoded reserved symbol or a presence of an unexpected encodedreserved symbol, is configured to initiate an error correction routine,wherein the arithmetic decoder is configured to not initiate the errorcorrection routine upon detection of an allowed symbol in the receiveddata, and wherein the sequential decoder is configured to, uponinitiation of the error correction routine by the arithmetic decoder,change at least one bistable element in the sequential decoder or selecta series of sequential bits from a plurality of series of sequentialbits which result from the changing of bistable elements in thesequential decoder.
 22. The system of claim 21, further comprising: thearithmetic encoder configured to receive input data; and a modulatorlocated between the arithmetic encoder and the channel.
 23. The systemof claim 21, wherein the decoder is configured for use in acommunication system.
 24. The system of claim 23, wherein thecommunication system is configured to use Binary Phase Shift Keying(BPSK), Quadrature Phase Shift Keying (QPSK), or Trellis CodedModulation (TCM).
 25. A method, comprising: receiving data from anencoder via a channel at a decoder, the decoder comprising an arithmeticdecoder and a sequential decoder; and producing output data using thedecoder, wherein the arithmetic decoder, upon detecting an absence of anexpected encoded reserved symbol or a presence of an unexpected encodedreserved symbol, initiates an error correction routine, wherein thearithmetic decoder does not initiate the error correction routine upondetection of an allowed symbol in the received data, and wherein thesequential decoder, upon initiation of the error correction routine bythe arithmetic decoder, changes at least one bistable element in thesequential decoder or selects a series of sequential bits from aplurality of series of sequential bits which result from the changing ofbistable elements in the sequential decoder.
 26. The method of claim 25,further comprising: receiving coded data from the encoder at amodulator, which outputs modulated data to the channel.
 27. The methodof claim 25, wherein the received data is a signal corresponding to abinary bit stream sequence of “1”/(“0”)'s and “0”/(“1”)'s.
 28. Themethod of claim 27, wherein the producing output data further comprisesmaking hard logic circuitry decisions as to a presence of the“1”/(“0”)'s and “0”/(“1”)'s based on the signal.
 29. The method of claim28, wherein the producing output data further comprises using thesequential decoder to monitor output from the arithmetic decoder forerrors.
 30. The method of claim 29, wherein the producing output datafurther comprises: identifying a branch point if the signal has a valuelocated within a null zone; and correcting the “1”/(“0”) or “0”/(“1”)binary bit by inverting it to “0”/(“1”) or “1”/(“0”) respectively, atthe sequential decoder if an output from the arithmetic decoderindicates an error.
 31. The method of claim 30, wherein the identifyinga branch point and the correcting the “1”/(“0”) or “0”/(“1”) binary bitis performed more than once.
 32. The method of claim 31, wherein theproducing output data further comprises determining a number of branchpoints at which correction of the “1”/(“0”) or “0”/(“1”) binary bit hasbeen performed (Kc).
 33. The method of claim 32, wherein the producingoutput data further comprises: decreasing a value of Kc by 1 if asubsequent branch point is sequentially prior to a first branch point;and increasing the value of Kc by 1 if the subsequent branch point isnot sequentially prior to the first branch point.
 34. The method ofclaim 32, further comprising: determining a Hamming distance threshold(Tc); and expanding the null zone to increase a number of branch points,if the number of branch points at which correction has been performed(Kc) exceeds the Hamming distance threshold (Tc).
 35. The method ofclaim 31, wherein the producing the output data further comprises:determining a Euclidean distance between the received data and theoutput data; and expanding the null zone to increase a number of branchpoints, if the Euclidean distance for subsequent calculations increasesfaster than a defined rate.
 36. The method of claim 25, wherein theproducing the output data further comprises: selecting a series ofsequential bits in the sequential decoder by, eliminating any series ofsequential bits which contains an encoded reserved symbol; applying ametric to at least two series of sequential bits which do not contain anencoded reserved symbol and determining which series is most likelycorrect; or applying an Euclidean metric to at least two series ofsequential bits which do not contain an encoded reserved symbol anddetermining which series is most likely correct.
 37. The method of claim25, further comprising using the decoder in a communication system. 38.The method of claim 37, further comprising using Binary Phase ShiftKeying (BPSK), Quadrature Phase Shift Keying (QPSK), or Trellis CodedModulation (TCM) in the communication system.
 39. A system, comprising:a decoder, comprising an encoded symbol decoder and a sequentialdecoder, configured to receive data from an arithmetic encoder via achannel, wherein the encoded symbol decoder, upon detecting an absenceof an expected encoded reserved symbol or a presence of an unexpectedencoded reserved symbol, is configured to initiate an error correctionroutine, wherein the encoded symbol decoder is configured to notinitiate the error correction routine upon detection of an allowedsymbol in the received data, and wherein the sequential decoder isconfigured to, upon initiation of the error correction routine by theencoded symbol decoder, change at least one bistable element in thesequential decoder or select a series of sequential bits from aplurality of series of sequential bits which result from the changing ofbistable elements in the sequential decoder.
 40. The system of claim 39,wherein the encoded symbol decoder is an arithmetic decoder.
 41. Thesystem of claim 39, further comprising: the arithmetic encoderconfigured to receive input data; and a modulator located between theencoder and the channel.
 42. The system of claim 39, wherein the decoderis configured for use in a communication system.
 43. The system of claim39, wherein the communication system is configured to use Binary PhaseShift Keying (BPSK), Quadrature Phase Shift Keying (QPSK), or TrellisCoded Modulation (TCM).
 44. A method, comprising: receiving data from anencoder via a channel at a decoder, comprising an encoded symbol decoderand a sequential decoder; and producing output data using the decoder,wherein the encoded symbol decoder, upon detecting an absence of anexpected encoded reserved symbol or a presence of an unexpected encodedreserved symbol, initiates an error correction routine, wherein theencoded symbol decoder does not initiate the error correction routineupon detection of an allowed symbol in the received data, and whereinthe sequential decoder, upon initiation of the error correction routineby the encoded symbol decoder, changes at least one bistable element inthe sequential decoder or selects a series of sequential bits from aplurality of series of sequential bits which result from the changing ofbistable elements in the sequential decoder.
 45. The method of claim 44,further comprising: receiving the data from the encoder via a modulator,which outputs data to the channel.
 46. The method of claim 44, furthercomprising using an arithmetic encoder as the encoder and an arithmeticdecoder as the encoded symbol decoder.
 47. The method of claim 44,further comprising using a signal corresponding to a binary bit streamsequence of “1”/(“0”)'s and “0”/(“1”)'s as the received data.
 48. Themethod of claim 47, wherein the producing output data further comprisesmaking hard logic circuitry decisions as to a presence of the“1”/(“0”)'s and “0”/(“1”)'s based on the signal.
 49. The method of claim48, wherein the producing output data further comprises using thesequential decoder to monitor output from the encoded symbol decoder forerrors.
 50. The method of claim 49, wherein the producing output datafurther comprises: identifying a branch point if the signal has a valuelocated within a null zone; and correcting the “1”/(“0”) or “0”/(“1”)binary bit by inverting it to “0”/(“1”) or “1”/(“0”) respectively, inthe sequential decoder if an output from the encoded symbol decoderindicates an error.
 51. The method of claim 50, wherein the identifyinga branch point and the correcting the “1”/(“0”) or “0”/(“1”) binary bitis performed more than once.
 52. The method of claim 51, wherein theproducing output data further comprises determining a number of branchpoints at which correction of the “1”/(“0”) or “0”/(“1”) binary bit hasbeen performed (Kc).
 53. The method of claim 52, wherein the producingoutput data further comprises: decreasing a value of Kc by 1 if asubsequent branch point is sequentially prior to a first branch point;and increasing the value of Kc by 1 if the subsequent branch point isnot sequentially prior to the first branch point.
 54. The method ofclaim 52, further comprising: determining a Hamming distance threshold(Tc); and expanding the null zone to increase a number of branch points,if the number of branch points at which correction has been performed(Kc) exceeds the Hamming distance threshold (Tc).
 55. The method ofclaim 51, wherein the producing output data further comprises:determining a Euclidean distance between the received data and theoutput data; and expanding the null zone to increase a number of branchpoints, if the Euclidean distance for subsequent calculations increasesfaster than a defined rate.
 56. The method of claim 51, wherein theproducing the output data further comprises: selecting a series ofsequential bits in the sequential decoder by, eliminating any series ofsequential bits which contains an encoded reserved symbol; applying ametric to at least two series of sequential bits which do not contain anencoded reserved symbol and determining which series is most likelycorrect; or applying an Euclidean metric to at least two series ofsequential bits which do not contain an encoded reserved symbol anddetermining which series is most likely correct.
 57. The method of claim44, further comprising using the decoder in a communication system. 58.The method of claim 57, further comprising using Binary Phase ShiftKeying (BPSK), Quadrature Phase Shift Keying (QPSK), or Trellis CodedModulation (TCM) in the communication system.